Apparatus for recognizing parking area for autonomous parking and method thereof

ABSTRACT

A vehicle parking assistance device includes an image sensing device, an artificial intelligence learning device, and a controller connected with the image sensing device and the artificial intelligence learning device. The controller is configured to obtain an image using the image sensing device, detect at least one parking line pair in the obtained image, detect a parking slot based on deep learning, detect a parking area based on the detected parking slot and the at least one detected parking pair, detect an entrance point for the parking area, and generate parking information based on the parking area and the entrance point.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from and the benefit of Korean PatentApplication No. 10-2020-0160344, filed on Nov. 25, 2020, which is herebyincorporated by reference for all purposes as if set forth herein.

BACKGROUND Field

Exemplary embodiments relate to technologies of recognizing a parkingarea for autonomous parking.

Discussion of the Background

With the development of technology, an autonomous parking technology forassisting a vehicle to park or exit a parking space has been introduced.For example, in an autonomous parking technology including a remotesmart parking assist (RSPA) system, when a user pushes a button on asmart key, a vehicle may park or exit a parking space by itself withoutuser involvement in the parking or the exiting of a parking space.

The above information disclosed in this Background section is only forenhancement of understanding of the background of the invention and,therefore, it may contain information that does not constitute priorart.

SUMMARY

To perform autonomous parking, there is a need for a vehicle (or asystem of the vehicle) to recognize a parking area. In this case, thesystem may recognize a parking area using an ultrasonic wave, but,because an object, such as a previously parked vehicle, a wall, or apillar, around a space where the vehicle wants to park is typicallypresent, it may be impossible for the system using ultrasonic waves toperform autonomous parking in a parking area where there is no objectaround the space. A technology of recognizing a parking area using animage may recognize a space around a vehicle although there is no objectaround the vehicle, but it is difficult to detect a parking linedesignating a boundary of a parking space due to factors such asreflection of light or shadow effects. Particularly, a parking lineindicating a parking area may fail to be accurately distinguished due toan auxiliary line that is not a parking space boundary or a merchandiseloading mark that may exist on a parking lot.

The exemplary embodiments described herein have been made to solve theabove-mentioned problems occurring in conventional autonomous vehicleparking technologies.

The technical problems to be solved by the present inventive concept arenot limited to the aforementioned problems, and any other technicalproblems not mentioned herein will be clearly understood from thefollowing description by those skilled in the art to which exemplaryembodiments pertain.

According to an aspect, a vehicle parking assistance device may includean image sensing device, an artificial intelligence learning device, anda controller connected with the image sensing device and the artificialintelligence learning device. The controller may be configured to obtainan image using the image sensing device, detect at least one parkingline pair in the obtained image, detect a parking area based on deeplearning, detect a parking line based on the detected parking slot andthe at least one detected parking line pair, detect an entrance pointfor the parking area, and generate parking information for autonomousparking based on the parking area and the entrance point.

According to another aspect, a vehicle parking assistance device mayinclude an image sensing device, an artificial intelligence learningdevice, and a controller connected with the image sensing device and theartificial intelligence learning device. The controller may beconfigured to obtain an image using the image sensing device, detect aparking slot in the obtained image based on deep learning by means ofthe artificial intelligence learning device, detect a parking line andan entrance point in the obtained image, and generate parkinginformation for autonomous parking based on the detected parking slot,the detected parking line, and the detected entrance point.

According to another aspect, a vehicle parking assistance device mayinclude an image sensing device and a controller connected with theimage sensing device. The controller may be configured to obtain animage using the image sensing device, detect a parking line in theobtained image, detect an entrance point included in a parking areabased on an amount of change in a pixel value of the detected parkingline, and generate information about the detected entrance point.

According to another aspect, a vehicle parking assistance device mayinclude an image sensing device, an artificial intelligence learningdevice, and a controller connected with the image sensing device and theartificial intelligence learning device. The controller may beconfigured to obtain an image using the image sensing device, detect aparking line in the obtained image, detect a plurality of entrance pointcandidate groups based on an amount of change in a pixel value of thedetected parking line, detect an entrance point having high confidenceamong the entrance point candidate groups based on deep learning usingthe artificial intelligence learning device, and generate informationabout the detected entrance point.

According to another aspect, a method may include obtaining an image,detecting at least one parking line pair in the obtained image,detecting a parking slot based on deep learning, detecting a parkingarea based on the detected parking slot and the at least one detectedparking line pair, detecting an entrance point for the detected parkingarea, and generating parking information for autonomous parking based onthe detected parking area and the entrance point.

According to another aspect, a method may include obtaining an image,detecting a parking slot in the obtained image based on deep learning,detecting a parking line and an entrance point in the obtained image,and generating parking information for autonomous parking based on thedetected parking slot, the detected parking line, and the detectedentrance point.

According to another aspect, a method may include obtaining an image,detecting a parking line in the obtained image, detecting an entrancepoint included in a parking area based on an amount of change in a pixelvalue of the detected parking line, and generating information about thedetected entrance point.

According to another aspect, a method may include obtaining an image,detecting a parking line in the obtained image, detecting a plurality ofentrance point candidate groups based on an amount of change in a pixelvalue of the detected parking line, detecting an entrance point havinghigh confidence among the entrance point candidate groups based on deeplearning, and generating information about the detected entrance point.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention, andtogether with the description serve to explain the principles of theinvention.

FIG. 1 is a functional block diagram of a vehicle system deviceaccording to various embodiments.

FIG. 2 illustrates parameters configuring parking information accordingto various embodiments.

FIG. 3 illustrates an operation for generating parking informationaccording to various embodiments.

FIG. 4 illustrates an operational flowchart of a vehicle system devicefor outputting parking information according to various embodiments.

FIG. 5 illustrates another operation for generating parking informationaccording to various embodiments.

FIG. 6 illustrates another operational flowchart of a vehicle systemdevice for outputting parking information according to variousembodiments.

FIG. 7 illustrates an operational flowchart of a vehicle system devicefor detecting a parking line candidate group according to variousembodiments.

FIG. 8 illustrates an operation for detecting a parking slot based on adeep learning classifier according to various embodiments.

FIG. 9A illustrates an operation for obtaining various types of parkingslot images according to various embodiments.

FIG. 9B illustrates an operation for obtaining various types of parkingslot images according to various embodiments.

FIG. 10 illustrates an operation for learning a parking slot image byway of a deep learning classifier according to various embodiments.

FIG. 11 illustrates a type of an entrance point according to variousembodiments.

FIG. 12 illustrates an operation for detecting an entrance pointaccording to various embodiments.

FIG. 13 illustrates an operational flowchart of a vehicle system devicefor outputting information about an entrance point according to variousembodiments.

FIG. 14 illustrates an operational flowchart of a vehicle system devicefor detecting an entrance point according to various embodiments.

FIG. 15 illustrates another operation for detecting an entrance pointaccording to various embodiments.

FIG. 16 illustrates another operational flowchart of a vehicle systemdevice for outputting information about an entrance point according tovarious embodiments.

FIG. 17 illustrates an operational flowchart of a vehicle system devicefor learning data for an entrance point according to variousembodiments.

FIG. 18 illustrates an operation for learning data for an entrance pointaccording to various embodiments.

With regard to description of drawings, the same or similar denotationsmay be used for the same or similar components.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

The invention is described more fully hereinafter with reference to theaccompanying drawings, in which embodiments of the invention are shown.This invention may, however, be embodied in many different forms andshould not be construed as limited to the embodiments set forth herein.Rather, these embodiments are provided so that the description of thevarious aspects is thorough, and will fully convey the scope of theinvention to those skilled in the art.

Various embodiments and terms used therein are not intended to limit thetechnical features described herein to particular embodiments, and itshould be construed as including various modifications, equivalents, oralternatives of a corresponding embodiment. With regard to descriptionof drawings, similar denotations may be used for similar or relatedcomponents. A singular form of a noun corresponding to an item mayinclude one item or a plurality of the items, unless context clearlyindicates otherwise. As used herein, each of the expressions “A or B,”“at least one of A and B,” “at least one of A or B,” “A, B, or C,” “atleast one of A, B, and C,” and “at least one of A, B, or C,” may includeany and all combinations of one or more of the items listed togetherwith a corresponding expression among the expressions. Terms as “1st”and “2nd,” or “first” and “second” may be used to distinguish acorresponding component from another, and do not limit the components inanother aspect (e.g., importance or order). If an element (e.g., a firstelement) is referred to, with or without the term “operatively” or“communicatively”, as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), it means thatthe element may be coupled with the other element directly (e.g.,wiredly), wirelessly, or via a third element.

As used herein, the term “module” used in various embodiments mayinclude a unit implemented in hardware, software, or firmware, and maybe interchangeably used with other terms, such as “logic,” “logicblock,” “part,” or “circuitry”. A module may be a single integralcomponent, or a minimum unit or part thereof, adapted to perform one ormore functions. For example, according to an embodiment, the module maybe implemented in the form of an application-specific integrated circuit(ASIC).

As customary in the field, some exemplary embodiments are described andillustrated in the accompanying drawings in terms of functional blocks,units, and/or modules. Those skilled in the art will appreciate thatthese blocks, units, and/or modules are physically implemented byelectronic (or optical) circuits, such as logic circuits, discretecomponents, microprocessors, hard-wired circuits, memory elements,wiring connections, and the like, which may be formed usingsemiconductor-based fabrication techniques or other manufacturingtechnologies. In the case of the blocks, units, and/or modules beingimplemented by microprocessors or other similar hardware, they may beprogrammed and controlled using software (e.g., microcode) to performvarious functions discussed herein and may optionally be driven byfirmware and/or software. It is also contemplated that each block, unit,and/or module may be implemented by dedicated hardware, or as acombination of dedicated hardware to perform some functions and aprocessor (e.g., one or more programmed microprocessors and associatedcircuitry) to perform other functions. Also, each block, unit, and/ormodule of some exemplary embodiments may be physically separated intotwo or more interacting and discrete blocks, units, and/or moduleswithout departing from the scope of the inventive concepts. Further, theblocks, units, and/or modules of some exemplary embodiments may bephysically combined into more complex blocks, units, and/or moduleswithout departing from the scope of the inventive concepts.

Various embodiments may be implemented as software (e.g., a program)including instructions that are stored in a machine-readable storagemedium (e.g., an internal memory or an external memory). For example,the machine may invoke at least one of one or more instructions storedin the storage medium and may execute the invoked instruction. This mayallow the machine to be operated to perform at least one functiondepending on the at least one invoked instruction. The one or moreinstructions may contain a code made by a compiler or a code executableby an interpreter. The machine-readable storage medium may be providedin the form of a non-transitory storage medium. Here, the term “non-rotransitory” simply means that the storage medium is a tangible deviceand does not include a signal (e.g., an electromagnetic wave), but thisterm does not differentiate between where data is semipermanently storedin the storage medium and where data is temporarily stored in thestorage medium.

According to an embodiment, a method according various embodimentsdisclosed herein may be included and provided in a computer programproduct. The computer program product may be traded as commoditiesbetween sellers and buyers. The computer program product may bedistributed in the form of a machine-readable storage medium (e.g., acompact disc read only memory (CD-ROM)) or may be distributed (e.g.,downloaded or uploaded) directly or online through an application storeor between two user devices. When distributed online, at least part ofthe computer program product may be at least temporarily stored in amachine-readable storage medium, such as a memory of the manufacturer'sserver, a server of the application store, or a relay server, and may betemporarily generated.

According to various embodiments, each (e.g., a module or program) ofthe above-mentioned components may include a single entity or aplurality of entities, and some of the plurality of entities may beseparately arranged in another component. According to variousembodiments, one or more components of the above-mentioned components oroperations may be omitted, or one or more other components or operationsmay be added. Alternatively or additionally, the plurality of components(e.g., modules or programs) may be integrated into one component. Insuch a case, the integrated component may one or more functions of eachof the plurality of components to be the same or similar to beingperformed by a corresponding component of the plurality of componentsbefore the integration. According to various embodiments, operationsperformed by modules, programs, or other components may be carried outsequentially, in parallel, repeatedly, or heuristically, or at least oneor more of the operations may be executed in a different order oromitted, or other operations may be added.

FIG. 1 is a functional block diagram of a vehicle system device 100according to various embodiments.

Referring to FIG. 1, the vehicle system device 100 may refer to a systemdevice loaded onto a vehicle. The vehicle system device 100 may performthe overall function (e.g., autonomous driving or autonomous parking) ofthe vehicle. The vehicle system device 100 may include an image sensingdevice 110, a controller 120, an artificial intelligence learning device130, and an output device 140. According to other embodiments, thevehicle system device 100 may exclude at least some (e.g., the outputdevice 140) among the components shown in FIG. 1 or may further includeanother component (e.g., a communication interface or a memory) which isnot shown in FIG. 1. The components included in the vehicle systemdevice 100 may refer to software (e.g., a program) implemented byinstructions as well as hardware components.

The image sensing device 110 may be used to obtain an image. Forexample, the image sensing device 110 may be a camera including at leastone of one or more lenses, an image sensor, an image signal processor,or a flash imaging device. According to an embodiment, the image sensingdevice 110 may obtain an image surrounding the vehicle. In this case,the image sensing device 110 may include 4-channel or more camerasmounted on the vehicle. The image surrounding the vehicle may be, forexample, a surround view, 360-degree image of an area around thevehicle. The vehicle system device 100 may detect a parking area (or aparking slot) and an entrance point for parking the vehicle in theparking slot, based on the obtained image data or the surround viewimage.

The controller 120 may execute, for example, software (e.g., a program)to control at least one other component (e.g., a hardware or softwarecomponent) of the vehicle system device 100 connected to the controller120 and may perform a variety of data processing tasks or calculations.According to an embodiment, as at least a part of data processing orcalculation performed, the controller 120 may store commands or datareceived from another component (e.g., the image sensing device 110, theartificial intelligence learning device 130, or the output device 140)in a volatile memory, may process the commands or data stored in thevolatile memory, and may store resultant data in a non-volatile memory.According to an embodiment, the controller 120 may include a mainprocessor (e.g., a central processing unit or an application processor)or an auxiliary processor (e.g., a graphic processing unit, an imagesignal processor, a sensor hub processor, a communication processor)operable independently or together with each other. For example, whenthe controller 120 includes the main processor and the auxiliaryprocessor, the auxiliary processor may be configured to use lower powerthan the main processor or specialize in a specified function. Theauxiliary processor may be implemented independently of the mainprocessor or as a part thereof.

According to embodiments, the controller 120 may perform the overallfunction of the vehicle system device 100 for outputting parkinginformation. For example, the controller 120 may obtain an image by wayof the image sensing device 110 and may detect a plurality of parkingline candidate groups in the obtained image. The controller 120 maydetect at least one parking line pair among the plurality of parkingline candidate groups. The controller 120 may detect a parking slot inan image obtained based on deep learning by way of the artificialintelligence learning device 130. The controller 120 may detect aparking area based on the at least one detected candidate parking linepair and the detected parking slot and may detect an entrance point ofthe detected parking area. The controller 120 may generate parkinginformation for autonomous parking based on the detected parking areaand the detected entrance point and may output the generated parkinginformation through the output device 140.

For another example, the controller 120 may obtain an image by way ofthe image sensing device 110 and may preprocess image data of theobtained image. The controller 120 may detect a parking slot in theimage based on deep learning by way of the artificial intelligencelearning device 130. The controller 120 may detect a parking line in theimage. The controller 120 may detect an entrance point of the parkingarea based on the detected parking line and the detected parking slot.The controller 120 may generate parking information for autonomousparking based on the detected parking slot, the detected parking line,and the detected entrance point and may output the generated parkinginformation through the output device 140.

According to embodiments, the controller 120 may perform the overallfunction of the vehicle system device 100 for outputting informationabout the entrance point. For example, the controller 120 may obtain animage by way of the image sensing device 110 and may detect a parkingline in the obtained image. The controller 120 may detect an entrancepoint for a parking area based on an amount of change in a pixel valueof the detected parking line. The controller 120 may generateinformation about the detected entrance point and may output thegenerated information through the output device 140.

For another example, the controller 120 may obtain an image by way ofthe image sensing device 110 and may detect a parking line in theobtained image. The controller 120 may detect an entrance pointcandidate group for a parking area based on an amount of change in apixel value of the detected parking line. The controller 120 may detectan entrance point having high confidence based on deep learning usingthe artificial intelligence learning device 130. The controller 120 mayoutput information about the detected entrance point.

The above-mentioned example describes that the controller 120 performseach operation for generating parking information in an integratedmanner. However, the vehicle system device 100 according to embodimentsmay separately include a parking line detector 122 for detecting aparking line, a parking slot detector 124 for detecting a parking slot,and an entrance point detector 126 for detecting an entrance point. Eachof these components may be a hardware device or software (a program)stored in a memory. As the respective components may operate separatelywithout operating as one module, a change in algorithm, learning of thealgorithm, or performance enhancement of the algorithm may be separatelyperformed.

The artificial intelligence learning device 130 may include a hardwarestructure specialized in processing an artificial intelligence model,for example, a neural processing unit (NPU). For another example, theartificial intelligence learning device 130 may be present as a separateprogram in a memory (not shown) of the vehicle system device 100. Theartificial intelligence model may be generated by machine learning. Suchlearning may be performed in the vehicle system device 100 itself inwhich artificial intelligence is performed or may be performed by way ofa separate server. A learning algorithm may include, for example, but isnot limited to, supervised learning, unsupervised learning,semi-supervised learning, or reinforcement learning. The artificialintelligence model may include a plurality of artificial neural networklayers. An artificial neural network may be, but is not limited to, oneof a deep neural network (DNN), a convolutional neural network (CNN), arecurrent neural network (RNN), a restricted Boltzmann machine (RBM), adeep belief network (DBN), a bidirectional recurrent deep neural network(BRDNN), deep Q-networks, two or more combinations thereof. Additionallyor alternatively, the artificial intelligence model may include asoftware structure, other than a hardware structure.

According to embodiments, the artificial intelligence learning device130 may learn the image for the parking slot by way of a deep learningclassifier and may distinguish the parking slot in the surround viewimage depending to the learned result. For another example, theartificial intelligence learning device 130 may learn an image for theentrance point by way of the deep learning classifier and may classifyone entrance point among entrance point candidate groups depending onthe learned result.

The output device 140 may include a hardware component for visually oraudibly providing information about the parking information or theentrance point. For example, the output device 140 may include adisplay, a hologram device, or a projector, and a control circuit forcontrolling the corresponding device. For another example, the outputdevice 140 may include an audio device (e.g., a speaker) capable ofconverting a sound into an electrical signal or converting an electricalsignal into a sound.

FIG. 2 illustrates parameters configuring parking information accordingto various embodiments.

Referring to FIG. 2, a vehicle system device 100 of FIG. 1 may obtain animage 200 using an image sensing device 110 of FIG. 1. The image 200 mayinclude, for example, a surround view, 360-degree image that surrounds avehicle. To perform autonomous parking, the vehicle system device 100may use at least one information among the following types ofinformation: a) an entrance point (e.g., 210-1, 210-2, 210-3, or 210-4),b) a parking line (e.g., 220-1, 220-2, 220-3, or 220-4), or c) a parkingslot (e.g., 230-1 or 230-2).

The entrance point may be used to control autonomous parking. FIG. 2illustrates that the entrance point is represented as a point where theparking line and the parking slot meet, but the entrance point accordingto various embodiments may be an area including a portion of the parkingarea. The entrance point may include location coordinates (e.g., x and ycoordinates) and direction information (e.g., an angle). For example,first information among the direction information of the entrance pointmay correspond to a direction of a parking line, and second informationamong the direction information of the entrance point may correspond toa direction of a parking slot. Thus, the vehicle system device 100 maydetermine a type (or form) of a parking area depending on a type (orform) of the entrance point. An example of the type of the entrancepoint will be described with reference to FIG. 11.

FIGS. 3 and 4 illustrate an operation for generating parking informationaccording to an embodiment.

Referring to FIG. 3, in operation 301, a vehicle system device 100 ofFIG. 1 may obtain a surround view, 360-degree image 300 that surrounds avehicle 350 using an image sensing device 110 of FIG. 1. Embodimentswhere the vehicle system device 100 uses the surround view, 360-degreeimage 300 are described for convenience of description, but the form ofthe image obtained by the image sensing device 110 is not limited to asurround view form.

In operation 302, the vehicle system device 100 may detect at least oneparking line pair in the image 300. The parking line pair may refer to,for example, two parking lines forming one parking slot. For example,the parking line 310-1 and the parking line 310-2 may form one parkingline pair, and the parking line 310-3 and the parking line 310-4 mayform another parking line pair. FIG. 3 illustrates an example ofdetecting only two parking line pairs, but the number of parking linesand the number of parking line pairs, which are detected by the vehiclesystem device 100, are not limited thereto.

According to an embodiment, the vehicle system device 100 may detect aplurality of parking line candidate groups in the image 300 to detect aparking line pair. For example, a controller 120 of FIG. 1 may performfiltering (e.g., Gaussian filtering) to remove noise due to raw data orthe surround view image obtained by way of the image sensing device 110and may extract edge data from the filtered image. The controller 120may determine a point determined as being a line in the image 300 as aline feature point. The line feature point may include, for example,location information (e.g., x and y coordinates) and directioninformation based on a gradient, in the image 300. The controller 120may perform line fitting for the determined line feature point. Forexample, the controller 120 may extract lines by clustering featurepoints, each of which has a similar direction and location, among thedetermined line feature points. The extracted lines (i.e., parkinglines) may include both end points (e.g., x and y coordinates) anddirection information. The controller 120 may determine two lines, whichare parallel to each other and are separated from each other by aspecified interval, among a plurality of parking line candidate groupsdetermined through the filtering, the feature point detection, and theline fitting as a parking line pair.

In operation 303, the vehicle system device 100 may detect at least oneparking slot (e.g., 320-1 or 320-2) based on deep learning. The parkingslot may be, for example, a space between parking line pairs, which mayrefer to an entrance section of a parking area. According to anembodiment, the controller 120 may learn various types of parking slotsby way of an artificial intelligence learning device 130 of FIG. 1 andmay detect an area corresponding to the parking slot in the image 300based on the learned result. The parking line pair and the parking slotmay form one parking area (or a parking area).

In operation 304, the vehicle system device 100 may detect an entrancepoint (e.g., 330-1, 330-2, 330-3, or 330-4). The entrance point mayrefer to a point where an end point of the parking slot and an end pointof the parking line pair meet. The entrance point may be represented as,for example, location coordinates (e.g., x and y coordinates) in theimage 300. The vehicle system device 100 may control autonomous parkingusing a location of the entrance point.

In operation 305, the vehicle system device 100 may output parkinginformation for autonomous parking. The parking information may includeat least one information among, for example, identification information(e.g., index 0 or index 1) about a parkable area, a location and anangle of the entrance point (e.g., 340-1, 340-2, 340-3, or 340-4), or atype (e.g., a parallel type, a perpendicular type, a diagonal type, or astepped type) of a parking slot.

FIG. 4 illustrates an operational flowchart of a vehicle system devicefor outputting parking information according to an embodiment. In thedescription below, operations included in the operational flowchart maybe performed by a vehicle system device 100 of FIG. 1 or may beperformed by components included in the vehicle system device 100. Forexample, a controller 120 of the vehicle system device 100 may controlother components (e.g., an image sensing device 110, an artificialintelligence learning device 130, and the output device 140) to performoperations of the operational flowchart.

Referring to FIG. 4, in operation 410, the controller 120 may obtain animage using the image sensing device 110. The obtained image mayinclude, for example, a surround view, 360-degree image of thatsurrounds a vehicle including the vehicle system device 100.

In operation 420, the controller 120 may detect a plurality of parkingline candidate groups in the obtained image. The parking line may referto, for example, a line having a specified direction in the image.

In operation 430, the controller 120 may detect a parking line pair inthe obtained image. For example, the controller 120 may determine twoparking lines, which are parallel to each other and have a specifiedinterval between them, among the plurality of parking line candidategroups as a parking line pair. According to an embodiment, thecontroller 120 may perform operation 420 and operation 430 at the sametime without separately performing operation 420 and operation 430.Operation 420 and operation 430 may be performed by a parking linedetector 122 of FIG. 1.

In operation 440, the controller 120 (e.g., a parking slot detector 124of FIG. 1) may detect a parking slot in the image based on deeplearning. For example, the controller 120 may extract an area betweenthe detected parking line pairs and may classify the extracted areausing an artificial intelligence learning device 130 of FIG. 1. Theclassification may be performed by, for example, a DNN-based deeplearning classifier. The controller 120 may determine whether the areaextracted through the classification is a parking slot. When the areaextracted through the classification corresponds to the parking slot,the controller 120 may determine what a type of the parking slot is. Thetype of the parking slot may include, for example, but is not limitedto, a parallel type, a perpendicular type, a diagonal type, and astepped type.

In operation 450, the controller 120 may detect a parking area. Forexample, the controller 120 may determine the parking area based on theparking slot and a parking line corresponding to the parking slot amongthe parking line candidate groups (or parking line pairs).

In operation 460, the controller 120 (e.g., an entrance point detector126 of FIG. 1) may detect an entrance point of the determined parkingarea. The entrance point may be represented as, for example, locationcoordinates and direction information. The direction information of theentrance point may correspond to a direction of the parking line pair.According to an embodiment, the entrance point may be determinedtogether when the parking area is detected. In this case, the controller120 may omit operation 460.

In operation 470, the controller 120 may output parking information forautonomous parking. For example, the controller 120 may deliver theparking information to another component of the vehicle system device100 for autonomous driving. For another example, the controller 120 maydisplay the parking information on an output device 140 of FIG. 1 suchthat a user may identify the parking area and the entrance point.

FIGS. 5 and 6 illustrate an operation for generating parking informationaccording to another embodiment.

Referring to FIG. 5, in operation 501, a vehicle system device 100 ofFIG. 1 may obtain an image 500 including a vehicle 550 using an imagesensing device 110 of FIG. 1. The image 500 may be, for example, asurround view, 360-degree image that surrounds the vehicle 550.

In operation 502, the vehicle system device 100 may detect at least oneparking slot (e.g., 510-1 or 510-2) based on deep learning. The parkingslot may refer to, for example, a section a vehicle enters in a parkingarea (or a parking slot). According to an embodiment, a controller 120of FIG. 1 may learn various types of parking slots by way of anartificial intelligence learning device 130 of FIG. 1 and may detect anarea corresponding to the parking slot and a type of the parking slotindicated by the area in the image 500 based on the learned result. Thecontroller 120 may generate parking slot information indicating thedetected type of the parking slot and location information (e.g., x andy coordinates).

In operation 503, the vehicle system device 100 may detect at least oneparking line (e.g., 520-1, 520-2, 520-3, or 520-4) in the image 500. Forexample, the controller 120 may perform filtering (e.g., Gaussianfiltering) to remove noise due to raw data or the surround view imageobtained by way of the image sensing device 110 and may extract edgedata from the filtered image. The controller 120 may determine a pointdetermined as being a line in the image 500 as a line feature point. Theline feature point may include, for example, location information (e.g.,x and y coordinates) and direction information based on a gradient, inthe image 500. The controller 120 may perform line fitting for thedetermined line feature point. For example, the controller 120 mayextract lines by clustering feature points, each of which has a similardirection and location, among the determined line feature points. Theextracted lines (i.e., parking lines) may include both end points (e.g.,x and y coordinates) and direction information. The controller 120 maydetermine two lines, which are parallel to each other and are separatedfrom each other at a specified interval, among the plurality of parkinglines determined through the filtering, the feature point detection, andthe line fitting as a parking line pair (e.g., 520-1 and 520-2, or 520-3and 520-4). The parking line pair and the parking slot may form oneparking area, which is designated by its respective pair of parkinglines.

In operation 504, the vehicle system device 100 may detect an entrancepoint (e.g., 530-1, 530-2, 530-3, or 530-4) for the parking slot. Theentrance point may refer to a point where an end point of the parkingslot and an end point of the parking line pair meet. The entrance pointmay be represented as, for example, location coordinates (e.g., x and ycoordinates) in the image 500. The vehicle system device 100 may controlautonomous parking using a location of the entrance point.

In operation 505, the vehicle system device 100 may output parkinginformation for autonomous parking. The parking information may includeat least one information among, for example, identification information(e.g., index 0 or index 1) about a parkable area, a location and anangle of the entrance point (e.g., 540-1, 540-2, 540-3, or 540-4), or atype (e.g., a parallel type, a perpendicular type, a diagonal type, or astepped type) of the parking slot.

FIG. 6 illustrates an operational flowchart of a vehicle system devicefor outputting parking information according to another embodiment. Inthe description below, operations included in the operational flowchartmay be performed by a vehicle system device 100 of FIG. 1 or may beperformed by components included in the vehicle system device 100. Forexample, a controller 120 of the vehicle system device 100 may controlother components (e.g., an image sensing device 110, an artificialintelligence learning device 130, and an output device 140 of FIG. 1) toperform operations of the operational flowchart.

Referring to FIG. 6, in operation 610, the controller 120 may obtain animage using the image sensing device 110. The obtained image mayinclude, for example, a surround view, 360-degree image that surrounds avehicle including the vehicle system device 100.

In operation 620, the controller 120 may preprocess image data. Theimage data may refer to, for example, raw data of the image obtained inoperation 610 of FIG. 6 or data of the surround view image. For example,the controller 120 may filter the image data to remove noise in theimage data. The controller 120 may perform a task for extracting edgedata from the filtered image data.

In operation 630, the controller 120 (e.g., a parking slot detector 124of FIG. 1) may detect a parking slot in the image based on deeplearning. For example, the controller 120 may recognize an areacorresponding to the parking slot in the image using the artificialintelligence learning device 130 and may classify a type of the parkingslot indicated by the recognized area. The classification may beperformed by, for example, a DNN-based deep learning classifier. Thetype of the parking slot may include, for example, but is not limitedto, a parallel type, a perpendicular type, a diagonal type, and astepped type.

In operation 640, the controller 120 (e.g., a parking line detector 122of FIG. 1) may detect a parking line in the obtained image. The parkingline may refer to, for example, a line having a specified direction inthe image. The controller 120 may extract feature points from theobtained image and may detect a parking line through line fitting forthe extracted feature points. The controller 120 may determine twoparking lines, which are parallel to each other and have a specifiedinterval by which these parking lines are separated from each other,among the plurality of detected parking lines as a parking line pair.

According to another embodiment, the controller 120 or the parking linedetector 122 may detect a parking line based on the detected parkingslot. For example, the controller 120 may detect a parking line extendedfrom the parking slot, depending on the type of the parking slot. Inthis case, the direction of the parking line may be based on thedetected type of the parking slot.

In operation 650, the controller 120 (e.g., an entrance point detector126 of FIG. 1) may detect an entrance point. The entrance point may berepresented as, for example, location coordinates and directioninformation. The direction information of the entrance point maycorrespond to a direction of the parking line. In an embodiment, thecontroller 120 may detect an entrance point using at least one of thedetected parking slot or the detected parking line. For example, theentrance point may be a point where the parking slot and the parkingline meet. For another example, the entrance point may be an end pointof the parking slot or the parking line meet.

In operation 660, the controller 120 may output parking information forautonomous parking. For example, the controller 120 may deliver theparking information to another component of the vehicle system device100 for autonomous driving. For another example, the controller 120 maydisplay the parking information on an output device 140 of FIG. 1 suchthat a user may identify the parking slot and the entrance point.

FIG. 7 illustrates an operational flowchart of a vehicle system devicefor detecting a parking line candidate group according to variousembodiments. Operations shown in FIG. 7 may be performed as an exampleof, for example, operation 420 of FIG. 4 or operation 640 of FIG. 6.Operations 720 to 740 among the operations shown in FIG. 7 may beperformed by a parking line detector 122 of FIG. 1.

Referring to FIG. 7, in operation 710, a controller 120 of FIG. 1 maypreprocess image data. The image data may refer to, for example, rawdata of an image obtained in operation 310 of FIG. 3 or operation 610 ofFIG. 6 or data of a surround view image. For example, the controller 120may filter the image data to remove noise in the image data. Thecontroller 120 may perform a task for extracting edge data from thefiltered image data.

In operation 720, the controller 120 may detect a line feature pointbased on the preprocessed image data. The line feature point may beplural in number. The line feature point may include locationinformation and direction information.

In operation 730, the controller 120 may perform line fitting for thedetected line feature point. For example, the controller 120 maygenerate a line by clustering feature points, each of which has asimilar direction and location, among the determined line featurepoints. The generated line may include location coordinates (e.g., x andy coordinates) and direction information (e.g., an angle) for both endpoints.

In operation 740, the controller 120 may detect a parking line candidategroup in the image through the generated line. For example, thecontroller 120 may determine lines, each of which has a specified lengthor is parallel to another line, among the previously determined lines asa parking line candidate group.

FIG. 8 illustrates an operation for detecting a parking slot based on adeep learning classifier according to various embodiments.

Referring to FIG. 8, a vehicle system device 100 (e.g., a controller120) of FIG. 1 may use an image (e.g., 810-1 or 810-2) including aparking slot as an input of a DNN-based parking slot classifier. Theimage including the parking slot may be extracted from an image (e.g.,200 of FIG. 2) previously obtained by an image sensing device 110 ofFIG. 1 or may be obtained additionally by the image sensing device 110after the vehicle system device 100 detects a parking line candidategroup (or a parking line pair).

Because the DNN-based parking slot classifier is in a state learnedthrough an image including various types of parking slots, it mayidentify whether the input image includes a parking slot and mayidentify whether the input image indicates any type of parking slot whenthe input image includes the parking slot. For example, when the firstimage 810-1 is input to the DNN-based parking slot classifier, thecontroller 120 (or an artificial intelligence learning device 130 ofFIG. 1) may recognize that the first image 810-1 indicates a parkingslot and may recognize that a type of the recognized parking slotindicates a diagonal type 820-1. As a similar example, when the secondimage 810-2 is input to the DNN-based parking slot classifier, thecontroller 120 may recognize that the second image 810-2 indicates aparking slot and may recognize that a type of the recognized parkingslot indicates a perpendicular type 820-2.

FIGS. 9A, 9B, and 10 illustrate an operation for learning a parking slotaccording to various embodiments. FIGS. 9A and 9B illustrate anoperation for obtaining a parking slot image. FIG. 10 illustrates anoperation for learning a parking slot image by way of a deep learningclassifier. A method for learning a parking slot image, which will bedescribed below, may be performed by way of an artificial intelligencelearning device 130 of a vehicle system device 100 of FIG. 1 or may beperformed by way of a separate server linked with the vehicle systemdevice 100. In this case, the vehicle system device 100 may detect aparking slot by receiving a learned result from the server.

Referring to FIG. 9A, the vehicle system device 100 (e.g., theartificial intelligence learning device 130) may obtain an image 900including at least one parking slot by way of an image sensing device110 of FIG. 1. The artificial intelligence learning device 130 mayobtain a plurality of images, each of which includes a parking slot,while changing a place (e.g., a parking lot), weather, and a type of aparking slot. The artificial intelligence learning device 130 maygenerate a local patch image to be used for learning in the imageincluding the parking slot. In an embodiment, the artificialintelligence learning device 130 may generate a plurality of local patchimages including a parking slot (e.g., 910-1) through pairing ofentrance points (e.g., 920-1 and 920-2). The local patch image mayinclude information about location coordinates (e.g., x and y), a width,a height, or a type (e.g., a perpendicular type, a parallel type, adiagonal type, a stepped type, and an open/closed type) of a parkingslot.

Referring to FIG. 9B, the type of the parking slot may include, forexample, but is not limited to, a closed perpendicular type (930-1), anopen perpendicular type (930-2), a closed parallel type 930-3, a closeddiagonal type 930-4, an open diagonal type 930-5, and a stepped type930-6.

Referring to FIG. 10, the artificial intelligence learning device 130may learn a local patch image 1005 by way of a deep learning classifierto accurately recognize a type of a parking slot in various situations(e.g., a place, weather, and the like). The artificial intelligencelearning device 130 may first determine whether the local patch image1005 obtained before learning is a parking slot and may then performlearning when the local patch image 1005 corresponds to the parkingslot. The deep learning classifier may learn a parking slot based onvarious types of artificial intelligence models. For example, the deeplearning classifier may enhance accuracy of image recognition bylearning a type of a parking slot using a CNN structure. For example, aninput 1010 of the deep learning classifier may be local patch imagesindicating various types of parking slots and a parking slot typeindicated by each image. The deep learning classifier may repeatedlyperform convolution and sub-sampling for data of the input 1010. Theconvolution may be understood as an operation of applying a mask to aninput image, multiplying a weight of the mask for each pixel value ofthe input image, and setting the sum thereof to a pixel value of anoutput image. The input 1010 of the deep learning classifier may beoutput as a plurality of images 1020 through the convolution. Theplurality of images 1020 may be referred to as a feature map. Thesub-sampling may be an operation of reducing a size of a screen, whichmay be referred to as a pooling operation. The sub-sampling may beunderstood as an operation of reducing a feature map of an M×M size to afeature map of an N×N size. A plurality of images 1030, each of whichhas the reduced size through the sum-sampling, may be generated. Whenthe above-mentioned convolution and the above-mentioned sub-sampling arerepeated, the image may increase in number and the image may decrease insize (1040 and 1050). The reduced images may have only features. Thefeature may be to distinguish, for example, a type of a parking slot,which may include a location of an entrance point, a direction of theentrance point, an angle of the entrance point, whether the entrancepoint is open/closed, or the like. The deep learning classifier maylearn a plurality of images 1050 generated through the convolution andthe sub-sampling by applying the plurality of images 1050 to at leastone hidden layer 1060 (e.g., a DNN). The deep learning classifier mayclassify a type of a parking slot in an image obtained thereafter usinga learned result 1070.

FIGS. 11 to 18 illustrate an operation for outputting information aboutan entrance point according to various embodiments.

FIG. 11 illustrates a type of an entrance point according to variousembodiments.

Referring to FIG. 11, a vehicle system device 100 of FIG. 1 may obtainor learn and store data 1101 to 1111 for various entrance point types.The entrance point type may be determined based on whether there is aparking slot and an angle where the parking slot and a parking linemeet. The entrance point type may include, for example, an open type1101, a T type 1102, a left L type 1103, a right L type 1104, a T type1105 slanted to the left, a left L type 1106 to be slanted to the left,a right L type 1107 slanted to the left, a T type 1108 slanted to theright, a left L type 1109 slanted to the right, a right L type 1110slanted to the right, and a U type 1111.

FIGS. 12 to 14 illustrate an operation for outputting information aboutan entrance point according to an embodiment.

Referring to FIG. 12, in operation 1201, a controller 120 of FIG. 1 maydetect a parking line 1210 from an obtained image 1200. In operation1202, the controller 120 may extract a profile 1220 for pixel values ofthe detected parking line 1210. An x-axis of a graph indicating theprofile 1220 may indicate an x-axis distance in the image 1200, and ay-axis of the graph may refer to a pixel value. An interval 1230 wherethe amount of change is large on the graph (or an interval where theamount of change is greater than a threshold) may refer to an interval1240 where the parking line is ended. The controller 120 may know thatthere is an entrance point at a location adjacent to the interval 1240where the parking line 1210 is ended, but a result for a location andtype of the entrance point may fail to be accurate when the interval1230 where the amount of change is large in width or when a differenceof the amount of change is not large.

To more accurately recognize the location and type of the entrancepoint, in operation 1203, the controller 120 may detect an entrancepoint through mask filtering for the interval (e.g., 1230 of FIG. 12)where the amount of change is large. The mask filtering may refer to ascheme of comparing a form for the interval where the amount of changeis large with data previously stored in a vehicle system device 100 ofFIG. 1. The previously stored data may refer to, for example, data forvarious entrance point types indicated in FIG. 11. The controller 120may compare the interval 1210 in the image 1200 corresponding to theinterval 1230 where the amount of change is large with previously storeddata to determine an entrance point type with the highest matching scoreas an entrance point type of the interval. The controller 120 may moreaccurately detect a location of the entrance point based on thepreviously detected parking line and the determined entrance point type.For example, in operation 1204, the controller 120 may determine a firstpoint 1250 on the graph as a location of an entrance point 1260 in theimage 1200.

FIG. 13 illustrates an operational flowchart of a vehicle system devicefor outputting information about an entrance point according to anembodiment. In the description below, operations included in theoperational flowchart may be performed by a vehicle system device 100 ofFIG. 1 or may be performed by components included in the vehicle systemdevice 100. For example, a controller 120 of the vehicle system device100 may control other components (e.g., an image sensing device 110 andan output device 140 of FIG. 1) to perform operations of the operationalflowchart. For another example, in operation 1320 may be performed by aparking line detector 122 of FIG. 1, and operation 1330 and operation1340 may be performed by an entrance point detector 126 of FIG. 1.

Referring to FIG. 13, in operation 1310, the controller 120 may obtainan image using the image sensing device 110. The obtained image mayinclude, for example, a surround, 360-degree view image that surrounds avehicle including the vehicle system device 100.

In operation 1320, the controller 120 may detect a parking line pairfrom the obtained image. For example, the controller 120 may performfiltering (e.g., Gaussian filtering) to remove noise due to raw data orthe surround view image obtained by way of the image sensing device 110and may extract edge data from the filtered image. The controller 120may determine a point determined as being a line in the image as a linefeature point. The line feature point may include, for example, locationinformation (e.g., x and y coordinates) and direction information basedon a gradient, in the image. The controller 120 may perform line fittingfor the determined line feature point. For example, the controller 120may extract lines by clustering feature points, each of which has asimilar direction and location, among the determined line featurepoints. The extracted lines (i.e., parking lines) may include both endpoints (e.g., x and y coordinates) and direction information.

In operation 1330, the controller 120 may detect an entrance point basedon an amount of change in a pixel value of the detected parking line.For example, an amount of change between pixel values corresponding tothe parking line is not large, whereas a pixel value at a point wherethe parking line is ended has a large difference with the pixel valuecorresponding to the parking line. Thus, the controller 120 maydetermine a point, where the amount of change is large, as an entrancepoint.

In operation 1340, the controller 120 may output information about theentrance point. The information about the entrance point may include atleast one of, for example, a location, an angle, a direction, or a typeof the entrance point. In an embodiment, the controller 120 may deliverthe information about the entrance point to another component of thevehicle system device 100 for autonomous driving. For another example,the controller 120 may display the information about the entrance pointon an output device 140 of FIG. 1 such that a user may identify theentrance point.

FIG. 14 illustrates an operational flowchart of a vehicle system devicefor detecting an entrance point according to an embodiment. Operationsshown in FIG. 14 may be implemented by a controller 120 or an entrancepoint detector 126 of FIG. 1.

Referring to FIG. 14, in operation 1410, the controller 120 may extracta profile for pixel values indicating a parking line from an image.

In operation 1420, the controller 120 may measure an amount of change onthe profile.

In operation 1430, the controller 120 may detect an interval where theamount of change is large. In an embodiment, the controller 120 maydetect an interval where the amount of change in pixel value is greaterthan a specified threshold.

In operation 1440, the controller 120 may detect an entrance pointthrough mask filtering for the interval (e.g., 530 of FIG. 5) where theamount of change is large. The mask filtering may refer to a scheme ofcomparing a form for the interval where the amount of change is largewith data previously stored in a vehicle system device 100 of FIG. 1.

FIGS. 15 to 18 illustrate an operation for outputting information aboutan entrance point according to another embodiment.

Referring to FIG. 15, in operation 1501, a controller 120 of FIG. 1 maydetect a parking line 1510 from an obtained image 1500. For example, thecontroller 120 may preprocess the obtained image 1500 and may detect theparking line 1510 through feature point extraction and line fitting ofthe preprocessed image 1500.

In in operation 1502, the controller 120 may extract a profile 1520 forpixel values of the detected parking line 1510. An x-axis of a graphindicating the profile 1520 may indicate an x-axis distance in the image1500, and a y-axis of the graph may refer to a pixel value. Thecontroller 120 may measure an amount of change in a pixel value and maydetect an interval (e.g., 1530) where the amount of change is greaterthan a specified threshold. The interval where the amount of change isgreater than the threshold may be plural in number on the profile 1520.In this case, the controller 120 may determine an interval as anentrance point candidate group.

According to an embodiment, the controller 120 may set the number ofentrance point candidate groups to N (where N is a natural number). Inthis case, the controller 120 may use N entrance point candidate groups,each of which has the largest amount of change, among the plurality ofentrance point candidate groups, each of which has the amount of changegreater than the threshold.

In in operation 1503, the controller 120 may extract an image (e.g.,1540-1, 1540-2, 1540-3, or 1540-4) for each of the entrance pointcandidate groups. In in operations 1504 to 1506, the controller 120 mayclassify an image for each of the entrance point candidate groups by wayof a classifier. For example, the controller 120 may compare the imagefor each of the entrance point candidate groups with data previouslylearned by the artificial intelligence learning device 130 to determinean image 1550 having the highest confidence as an image corresponding tothe entrance point. The classifier may determine a location and type ofan entrance point 1560 included in the image 1500 by comparing the imageof the entrance point candidate group with previously learned data.

FIG. 16 illustrates an operational flowchart of a vehicle system devicefor outputting information about an entrance point according to anotherembodiment. In the description below, operations included in theoperational flowchart may be performed by a vehicle system device 100 ofFIG. 1 or may be performed by components included in the vehicle systemdevice 100. For example, a controller 120 of the vehicle system device100 may control other components (e.g., an image sensing device 110, anartificial intelligence learning device 130, and an output device 140 ofFIG. 1) to perform operations of the operational flowchart. For anotherexample, operation 1620 may be performed by a parking line detector 122of FIG. 1, and operations 1630 to 1650 may be performed by an entrancepoint detector 126 of FIG. 1.

Referring to FIG. 16, in operation 1610, the controller 120 may obtainan image using the image sensing device 110. The obtained image mayinclude, for example, a surround view 360-degree image that surrounds avehicle including the vehicle system device 100.

In operation 1620, the controller 120 may detect a parking line from theobtained image. For example, the controller 120 may perform filtering(e.g., Gaussian filtering) to remove noise due to raw data or thesurround view image obtained by way of the image sensing device 110 andmay extract edge data from the filtered image. The controller 120 maydetermine a point determined as being a line in the image as a linefeature point. The line feature point may include, for example, locationinformation (e.g., x and y coordinates) and direction information basedon a gradient, in the image. The controller 120 may perform line fittingfor the determined line feature point. For example, the controller 120may extract lines by clustering feature points, each of which has asimilar direction and location, among the determined line featurepoints. The extracted lines (i.e., parking lines) may include both endpoints (e.g., x and y coordinates) and direction information.

In operation 1630, the controller 120 may detect an entrance pointcandidate group based on an amount of change in a pixel value of thedetected parking line. For example, an amount of change between pixelvalues corresponding to the parking line is not large, whereas a pixelvalue at a point where the parking line is ended has a large differencewith the pixel value corresponding to the parking line. Thus, thecontroller 120 may determine a point, where the amount of change islarge, as an entrance point candidate group.

In operation 1640, the controller 120 may detect an entrance pointhaving high confidence among the entrance point candidate groups basedon deep learning. For example, the controller 120 may compare datadetermined as the entrance point candidate group with data learned bythe artificial intelligence learning device 130 and may select anentrance point candidate group having high confidence as a result of thecompared result.

In operation 1650, the controller 120 may output information about theentrance point. The information about the entrance point may include atleast one of, for example, a location, an angle, a direction, or a typeof the entrance point. In an embodiment, the controller 120 may deliverthe information about the entrance point to another component of thevehicle system device 100 for autonomous driving. For another example,the controller 120 may display the information about the entrance pointon an output device 140 of FIG. 1 such that a user may identify theentrance point.

FIG. 17 illustrates an operational flowchart of a vehicle system devicefor learning data for an entrance point according to variousembodiments.

Referring to FIG. 17, in operation 1710, an artificial intelligencelearning device 130 of FIG. 1 may collect an image including an entrancepoint.

In operation 1720, the artificial intelligence learning device 130 maylearn an image collected through a deep learning classifier. The deeplearning classifier may use at least one scheme among, for example,multilayer perception (MLP), support vector machine (SVM), or a deepneural network (DNN).

FIG. 18 illustrates an operation for learning data for an entrancepoint.

A method for learning data for an entrance point, which will bedescribed below, may be performed by way of an artificial intelligencelearning device 130 of a vehicle system device 100 of FIG. 1 or may beperformed by way of a separate server linked with the vehicle systemdevice 100. In this case, the vehicle system device 100 may detect anentrance point by receiving a learned result from the server.

The artificial intelligence learning device 130 may obtain and collectimages, each of which includes the entrance point, by way of an imagesensing device 110 of FIG. 1. The artificial intelligence learningdevice 130 may obtain a plurality of images, each of which includes anentrance point, while changing a place (e.g., a parking lot), weather,and a type of the entrance point. The entrance point type may be anexample shown in FIG. 11, but not limited thereto.

The artificial intelligence learning device 130 may generate a localpatch image to be used for learning in the image including the entrancepoint and may learn the local patch image by means of the deep learningclassifier. The deep learning classifier may learn data for the entrancepoint based on various types of artificial intelligence models. Forexample, the deep learning classifier may enhance accuracy of imagerecognition by learning a type of the entrance point using a CNNstructure. A process (1810 to 1870) where the data for the entrancepoint is learned by the deep learning classifier may be similar inprinciple to the process (1010 to 1070) where the data for the parkingslot is learned in FIG. 10. In this case, an input 1810 of the deeplearning classifier may be local patch images indicating various typesof entrance points and an entrance point type indicated by each image.Furthermore, a feature indicated by reduced images may be to distinguishan entrance point, which may include a location, a direction, an angle,a form, or the like of the entrance point. The deep learning classifiermay classify an entrance point with high confidence among the entrancepoint candidate groups using a learned result 1870.

According to embodiments disclosed herein, the vehicle system device maymore accurately recognize a parking area where there are no objectsaround the parking area.

According to embodiments disclosed herein, the vehicle system device maymore accurately recognize an entrance point in a parking area wherethere are no objects around the parking area.

In addition, various effects ascertained directly or indirectly throughthe embodiments disclosed herein may be provided.

Hereinabove, although exemplary embodiments have been described withreference to the accompanying drawings, these embodiments are notlimited thereto, but may be variously modified and altered by thoseskilled in the art to which the exemplary embodiments pertain withoutdeparting from the spirit and scope of the invention as set forth in thefollowing claims.

What is claimed is:
 1. A vehicle parking assistance device, comprising:an image sensing device; an artificial intelligence learning device; anda controller connected with the image sensing device and the artificialintelligence learning device, wherein the controller is configured to:obtain an image using the image sensing device; detect at least oneparking line pair in the obtained image; detect a parking slot based ondeep learning; detect a parking area based on the detected parking slotand the at least one detected parking line pair that correspond to firstand second boundaries of the detected parking slot; detect an entrancepoint for the parking area; and generate parking information forautonomous parking of a vehicle based on the parking area and theentrance point.
 2. The vehicle parking assistance device of claim 1,wherein the controller is configured to: obtain a surround view,360-degree image of an area surrounding the vehicle using the imagesensing device.
 3. The vehicle parking assistance device of claim 1,wherein the controller is configured to: detect a plurality of parkingline candidate groups in the obtained image; and detect the at least oneparking line pair which is parallel to each other and has a specifiedinterval among the plurality of parking line candidate groups.
 4. Thevehicle parking assistance device of claim 3, wherein the controller isconfigured to: preprocess image data of the obtained image; detect aline feature point from the preprocessed image data; perform linefitting for the detected line feature point; and detect the plurality ofparking line candidate groups based on the line fitting.
 5. The vehicleparking assistance device of claim 1, further comprising: an outputdevice, wherein the controller is configured to: output the parkinginformation through the output device.
 6. The vehicle parking assistancedevice of claim 1, wherein the controller is configured to: detect,using an artificial learning device, the parking slot based on a deepneural network (DNN).
 7. The vehicle parking assistance device of claim1, wherein the parking information includes at least one ofidentification information about a parkable area, a location and anangle of the entrance point, or a type of the parking slot.
 8. Thevehicle parking assistance device of claim 1, wherein detect a parkingslot based on deep learning comprises using an artificial intelligencelearning device to detect the parking slot.
 9. The vehicle parkingassistance device of claim 1, wherein the parking slot comprises atleast one of a parallel type, a perpendicular type, a diagonal type, ora stepped type of a parking slot.
 10. A method comprising: obtaining animage; detecting at least one parking line pair in the obtained image;detecting a parking slot based on deep learning; detecting a parkingarea based on the detected parking slot and the at least one detectedparking line pair that correspond to first and second boundaries of thedetected parking slot; detecting an entrance point for the parking area;and generating parking information for autonomous parking of a vehiclebased on the parking area and the entrance point.
 11. The method ofclaim 10, wherein the obtaining of the image includes obtaining asurround view, 360-degree image surrounding the vehicle.
 12. The methodof claim 10, further comprising: detecting a plurality of parking linecandidate groups in the obtained image, wherein the detecting of the atleast one parking line pair includes detecting the at least one parkingline pair which is parallel to each other and has a specified intervalamong the plurality of parking line candidate groups.
 13. The method ofclaim 12, wherein the detecting of the plurality of parking linecandidate groups includes: preprocessing image data of the obtainedimage; detecting a line feature point from the preprocessed image data;performing line fitting for the detected line feature point; anddetecting the plurality of parking line candidate groups based on theline fitting.
 14. The method of claim 10, further comprising: outputtingthe generated parking information.
 15. The method of claim 10, whereinthe detecting of the parking slot based on the deep learning includes:detecting the parking slot based on a DNN.
 16. The method of claim 10,wherein the parking information includes at least one of identificationinformation about a parkable area, a location and an angle of theentrance point, or a type of the parking slot.
 17. The method of claim10, wherein detecting a parking slot comprises performing deep learningusing an artificial intelligence learning device to detect the parkingslot.
 18. The method of claim 10, wherein the parking slot comprises atleast one of a parallel type, a perpendicular type, a diagonal type, ora stepped type of a parking slot.
 19. A non-transitory computer readableprogram medium comprising program code, that when executed by at leastone processor, cause the at least one processor to perform operationscomprising: obtaining an image; detecting at least one parking line pairin the obtained image; detecting a parking slot based on deep learning;detecting a parking area based on the detected parking slot and the atleast one detected parking line pair that correspond to first and secondboundaries of the detected parking slot; detecting an entrance point forthe parking area; and generating parking information for autonomousparking of a vehicle based on the parking area and the entrance point.20. The non-transitory computer readable program medium of claim 19,wherein the program code, when executed by the at least one processor,further causes the at least one processor to perform operationscomprising: obtaining a surround view, 360-degree image surrounding avehicle using an image sensing device.